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Abstract: We discuss the long-run behavior of stochastic dynamics of many interacting players in 
spatial evolutionary games. In particular, we investigate the effect of the number of players and 
the noise level on the stochastic stability of Nash equilibria. We discuss similarities and differences 
between systems of interacting players maximizing their individual payoffs and particles minimizing 
their interaction energy. We use concepts and techniques of statistical mechanics to study game- 
theoretic models. In order to obtain results in the case of the so-called potential games, we analyze 
the thermodynamic limit of the appropriate models of interacting particles. 
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1. Introduction 

Many socio-economic systems and biological populations can be modeled as systems of interacting 
individuals [1—5]. Here we will consider game-theoretic models of many interacting players [6-8]. In 
such models, individuals have at their disposal certain strategies and their payoffs in a game depend 
on strategies chosen both by them and by their opponents. In spatial games, players are located on 
vertices of certain graphs and they interact only with their neighbors [2,9-17]. The central concept 
in game theory is that of a Nash equilibrium. It is an assignment of strategies to players such that no 
player, for fixed strategies of his opponents, has an incentive to deviate from his curent strategy; the 
change can only diminish his payoff. 

The notion of a Nash equilibrium (called a Nash configuration in spatial games) is similar to the 
notion of a ground-state configuration in classical lattice-gas models of interacting particles. We will 
discuss similarities and differences between systems of interacting players maximizing their individual 
payoffs and particles minimizing their interaction energy. 

One of the fundamental problems in game theory is the equilibrium selection in games with multiple 
Nash equilibria. One of the selection methods is to construct an appropriate dynamical system where 
in the long run only one equilibrium is played with a high frequency. Here we will discuss a stochastic 
adaptation dynamics of a population with a fixed number of players. In discrete moments of times, 
players adapt to their neighbors by choosing with a high probability the strategy which is the best 
response, i.e. the one which maximizes the sum of the payoffs obtained from individual games. With 
a small probability, representing the noise of the system, they make mistakes. To describe the long- 
run behavior of such stochastic dynamics, Foster and Young [18] introduced a concept of stochastic 
stability. A configuration of the system (an assignment of strategies to lattice sites in spatial games) 
is stochastically stable if it has a positive probability in the stationary state of the above dynamics in 
the zero- noise limit, that is zero probability of mistakes. It means that in the long run we observe it 
with a positive frequency. However, for any arbitrarily low but fixed noise, if the number of players 
is big enough, the probability of any individual configuration is practically zero. It means that for a 
large number of players, to observe a stochastically stable configuration we must assume that players 
make mistakes with extremely small probabilities. On the other hand, it may happen that in the long 
run, for a low but fixed noise and sufficiently big number of players, the stationary state is highly 
concentrated on an ensemble consisting of one Nash configuration and its small perturbations, i.e. 
configurations, where most players play the same strategy. We will call such configurations ensemble 
stable. We will show that these two stability concepts do not necessarily coincide. 
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In the so-called potential games, for any given configuration, payoffs of all players are the same [19]. 
Such systems are therefore analogous to those of interacting particles, where instead of maximizing 
payoffs, particles minimize their interaction energy. Stationary states of the stochastic dynamics with 
the Boltzmann-type updating are then finite-volume Gibbs distributions describing an equilibrium 
behavior of corresponding systems of interacting particles in the grand-canonical ensemble. We use 
techniques and results of statistical mechanics to describe the long-run behavior of potential games. 
We investigate a thermodynamic limit, i.e. the limit of the infinite number of players. 

We will present examples of spatial games with three strategies where concepts of stochastic stability 
and ensemble stability do not coincide. In particular, we may have the situation, where a stochastically 
stable strategy is played in the long run with an arbitrarily low frequency. 

We will also discuss briefly nonpotential games. Stationary states of such games cannot be ex- 
plicitly constructed as before. We must therefore resort to different methods. We will use a tree 
characterization of stationary states [20,21]. 

In Section 2, we introduce spatial games with local interactions. In Section 3, we present stochastic 
dynamics and the concept of stochastic stability of Nash configurations. In Section 4, we introduce 
our concept of ensemble stability and present examples of games where stochastically stable Nash 
configurations are played in the long run with arbitrarily small probabilities if the noise level is low 
and the number of players is big enough. We will also discuss an effect of adding a dominated strategy 
to a game with two strategies. In particular, the presence of such a strategy may cause a stochastically 
stable strategy to be observed in the long run with a frequency close to zero. In Section 5, we discuss 
the long-run behavior of a certain example of a nonpotential game. Discussion follows in Section 6. 

2. Spatial Games with Local Interactions 

In order to characterize a game-theoretic model, one has to specify the set of players, strategies 
they have at their disposal and payoffs they receive. Here we will discuss only two-player games with 
two or three pure strategies. In addition, players may use mixed strategies. A mixed strategy is a 
probability distribution on the set of pure strategies. We begin with games with two pure strategies 
and two symmetric Nash equilibria. A generic payoff matrix is given by 
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Example 1 

A B 

A a b 

U = 

Bed, 

where the ij entry, i,j = A, B, is the payoff of the first (row) player when he plays the strategy i and 
the second (column) player plays the strategy j. We assume that both players are the same and hence 
payoffs of the column player are given by the matrix transposed to U; such games are called symmetric. 
Let (x, 1 — x) be a mixed strategy, where x is the probability of playing A and 1 — x of playing B. We 
then assume that the payoff received by a player using a mixed strategy (x, 1 — x) against a player 
using (y, 1 — y) is the average (expected) payoff given by x[ay + b(l — y)\ + (1 — x)[{cy + d(l — y)\. 

An assignment of strategies to both players is a Nash equilibrium, if for each player, for a fixed 
strategy of his opponent, changing the current strategy will not increase his payoff. 

We will discuss games with multiple Nash equilibria. If a > c and d > b, then both (A, A) and 
(B, B) are Nash equilibria. If a + b < c + d, then the strategy B has a higher expected payoff against a 
player playing both strategies with the probability 1/2. We say that B risk dominates the strategy A 
(the notion of the risk-dominance was introduced and thoroughly studied by Harsanyi and Selten [22] ) . 
If at the same time a > d, then we have a selection problem of choosing between the payoff-dominant 
(Pareto-efficient) equilibrium (A, A) and the risk-dominant (B,B). 

We will study populations with a finite number of individuals playing two-player games. In spatial 
games, players occupy sites of certain lattices and interact only with their neighbors. 

Let A be a finite subset of the simple lattice Z 2 (for simplicity of presentation we assume periodic 
boundary conditions, i.e. we place players on a two-dimensional torus). Every site of A is occupied 
by one player who has at his disposal one of k different pure strategies (player do not use mixed 
strategies). Let S be the set of pure strategies, then Qa = S A is the set of all configurations of players, 
that is all possible assignments of strategies to individual players. For every i <G A, X; L is the strategy 
of the i— th player in the configuration X G Qa and A_j denotes strategies of all remaining players; 
X therefore can be represented as the pair (JQ,X_j). Let U : S x S ^ R be & matrix of payoffs 
of our game. Every player interacts only with his neighbors and his payoff is the sum of the payoffs 
resulting from individual games. We assume that he has to use the same strategy for all neighbors. 
Let Ni denote the neighborhood of the i— th player. For the nearest-neighbor interaction we have 
Ni = {j; \ j ' — i\ = 1}, where \i — j\ is the distance between i and j. For X G we denote by Vi{X) 
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the payoff of the i— th player in the configuration X: 

v i {X) = Y J U{X i ,X j ) (2.1) 

jeNi 

Definition Lie Q\ is a Nash configuration if for every i G A andYi <G S, > z^(Yj,X_j). 

In Example 1, there are two homogeneous Nash configurations, X A and X B , in which all players 
play the same strategy, A or B respectively. 

Let us notice that the notion of a Nash configuration is similar to the notion of a ground-state 
configuration in classical lattice-gas models of interacting particles. We have to identify agents with 
particles, strategies with types of particles and instead of maximizing payoffs we should minimize 
interaction energies. There are however profound differences. First of all, ground-state configurations 
can be defined only for symmetric matrices; an interaction energy is assigned to a pair of particles, 
payoffs are assigned to individual players. It may happen that if a player switches a strategy to 
increase his payoff, the payoff of his opponent and of the entire population decreases. Moreover, 
ground-state configurations are stable with respect to all local changes, not just one-site changes like 
Nash configurations. It means that for the same symmetric matrix U, there may exist a configuration 
which is a Nash configuration but not a ground-state configuration for the interaction marix —U. The 
simplest example is given by Example 1 with a = 2,b = c = 0, and d = 1. X A and X B are Nash 
configurations but only X A is a ground-state configuration for —U. 

For any classical lattice-gas model there exists at least one ground-state configuration. It may 
happen that a game with a nonsymmetric payoff matrix may not posess a Nash configuration. The 
classical example is that of the Rock-Scissors-Paper game given by the following matrix. 

Example 2 





R 


S 


P 


R 


1 


2 





S 





1 


2 


P 


2 





1 



This two-player game does not have a Nash equilibrium in pure strategies. It has a unique mixed 
Nash equilibrium in which both players use a mixed strategy, playing all three pure strategies with 
the probability 1/3. One may show that the game does not have any Nash configurations on Z and Z 2 
with nearest-neighbor interactions but it has multiple Nash configurations on the triangular lattice. 

In short, ground-state configurations minimize the total energy of a particle system, Nash configu- 
rations do not necessarily maximize the total payoff of a population of agents. 
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3. Stochastic Stability 

We describe now the deterministic dynamics of the best-response rule. Namely, at each discrete 
moment of time t = 1,2, a randomly chosen player may update his strategy. He simply adopts the 
strategy, Xj, which gives him the maximal total payoff Vi(Xf, X^ 1 ) for given X^ 1 , a configuration 
of strategies of remaining players at time t — 1. 

Now we allow players to make mistakes with a small probability, that is to say they may not 
choose the best response. A probability of making a mistake may depend on the state of the system (a 
configuration of strategies of neighboring players). We will assume that this probability is a decreasing 
function of the payoff lost as a result of a mistake [9]. In the Boltzmann-type updating (called a log- 
linear rule in the economics/game theory literature), the probability of chosing by the i— th player the 
strategy Xj at time t is given by the following conditional probability: 

(l/T)v t (XlX^) 

where T > measures the noise level. 

Let us observe that if T — > 0, pf converges to the best-response rule. Our stochastic dynamics 
is an example of an ergodic Markov chain with \S^\ states. Therefore, it has a unique stationary 
distribution (a stationary state) which we denote by fi\. 

The following definition was first introduced by Foster and Young [18]. 

Definition 2. X € fl\ is stochastically stable if lim^^o MaPO > 0. 

If X is stochastically stable, then the frequency of visiting X converges to a positive number along 
any time trajectory almost surely. It means that in the long run we observe X with a positive 
frequency. In examples below, we consider games with symmetric Nash equilibria and homogeneous 
Nash configurations. By a stochastic stability of a strategy or a Nash equilibrium we mean a stochastic 
stability of the corresponding Nash configuration. 

The notion of a stochastically stable Nash configuration is analogous to the notion of a low- 
temperature stable ground-state configurations, i.e. the one which gives rise to a low-temperature 
equilibrium phase. 

Stationary distributions of the Boltzmann dynamics can be explicitly constructed for the so-called 
potential games. A game is called a potential game if its payoff matrix can be changed to a symmetric 
one by adding payoffs to its columns. Such a payoff transformation does not change the strategic 
character of the game, in particular it does not change the set of its equilibria and their stochastic 
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stability. More formally, it means that there exists a symmetric matrix V called a potential of the 
game such that for any three strategies A,B,C G S 

U(A, C) - U(B, C) = V(A, C) - V(B, C). (3.2) 

It is easy to see that every game with two strategies has a potential V with V(A, A) = a — c, 
V(B,B) = d-b, and V(A,B) = V(B,A) = 0. If V is a potential of the stage game, then V{X) = 
J2(ij)£A Xj) is a potential of a configuration X in the corresponding spatial game. The unique 

stationary state of a potential game with the Boltzmann dynamics is given by the following formula [2]: 

T e (i/T)E W ) e A^,^) 

= v e ^m^ J)eA v(z^ Z] )- ( 3 - 3 ) 

n\ is a so-called finite-volume Gibbs state - a probability distribution describing an equilibrium 
behavior of a system of particles with a two-body Hamiltonian — V and the temperature T. The 
limit limx^o A*a * s a ground-state measure supported by ground-state configurations, that is Nash 
configurations with the biggest V. It follows from (3.3) that stochastically stable Nash configurations 
are those with the biggest potential. In particular, in spatial games with two strategies and two Nash 
equilibria, the risk-dominant configuration X A is stochastically stable. 

In Section 4, using statistical mechanics methods, we will study the behavior of n\ in the limit 
of the infinite number of players, i.e. in the thermodynamic limit, for various two-player games with 
three pure strategies. 

4. Ensemble Stability 

The concept of stochastic stability involves individual configurations of players. In the zero-noise 
limit, a stationary state is usually concentrated on one or at most few configurations. However, for a 
low but fixed noise and for a big number of players, the probability of any individual configuration of 
players is practically zero. The stationary state, however, may be highly concentrated on an ensemble 
consisting of one Nash configuration and its small perturbations, i.e. configurations, where most 
players use the same strategy. Such configurations have relatively high probability in the stationary 
state. We call such configurations ensemble stable. 

Definition 3. X G £l\ is e-ensemble stable if jJ^iY G ^a; Y% / Xi) < e for any i G A if A D A(T) 
for some A(T). 



Definition 4. X G 17a is low- noise ensemble stable if for every e > there exists T(e) such that 
ifT< T(e), then X is e-ensemble stable. 
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If X is e-ensemble stable with e close to zero, then the ensemble consisting of X and configurations 
which are different from X at at most few sites has the probability close to one in the stationary 
state. It does not follow, however, that X is necessarily low-noise ensemble or stochastically stable as 
it happens in examples presented below. 

Example 3 

Players are located on a finite subset A of Z 2 (with periodic boundary conditions) and interact with 
their four nearest neighbors. They have at their disposal three pure strategies: A, B, and C. The 
payoffs are given by the following symmetric matrix: 







A 


B 


C 




A 


1.5 





1 


u = 


B 





2 


1 




C 


1 


1 


2 



Our game has three Nash equilibria, (A, A), (B, B), and (C, C), and the corresponding spatial game 
has three homogeneous Nash configurations: X A ,X B , and X c . Let us notice that X B and X c have 
the maximal payoff in every finite volume and therefore they are ground-state configurations for — U 
and X A is not. 

The unique stationary state of the Boltzmann dynamics (3.1) is a finite- volume Gibbs state and 
is given by (3.3) with V replaced by U. £ (ij)eA U(X?, X*) - £ (iJ)6A £/(Y;, Y,) > 0, for every Y + 
X B andX c , k = B,C, and £ (iJ)6A U(X B ,X B ) = £ (iJ)eA U(X?, Xf). It follows that lim T ^ nl{X k ) = 
1/2, k = B,C so X B and X c are stochastically stable. Let us investigate the long-run behavior of 
our system for large A, that is for a big number of players. Observe that lim A ^z2 fj^(X) = for 
every X € = S z . Hence for large A and T > we may only observe, with reasonable positive 
frequencies, ensembles of configurations and not particular configurations. We will be interested in 
ensembles which consist of a Nash configuration and its small perturbations, that is configurations, 
where most players use the same strategy. We perform first the limit A — ► Z 2 and obtain a so-called 
infinite-volume Gibbs state in the temperature T, 

M T =lim^. (4.1) 
A^Z 2 

It describes, in the thermodynamic limit, the equilibrium behavior of a system of interacting par- 
ticles. Equilibrium behavior of such system results from the competition between its energy U and 
entropy S, i.e. the minimization of their free energy F = U — TS. We will show that it is the entropy 
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which is responsible for the ensemble stability of some Nash configurations (ground-state configura- 
tions) in the limit of the infinite number of players (lattice sites). The phase transition of the first 
kind is manifested by the existence of multiple Gibbs states for a given Hamiltonian and temperature. 

In order to investigate the stationary state of our example, we will apply a technique developed 
by Bricmont and Slawny [23,24]. They studied low-temperature stability of the so-called dominant 
ground-state configurations. It follows from their results that 

^(Xi = C) > 1 - e(T) (4.2) 

for any i £ Z 2 and e(T) — > as T — > 0. 

We will recall in Appendix A their proof adapted to our model. The following theorem is a simple 
consequence of (4.2). 

Theorem 1. X c is low-noise ensemble stable. 

We see that for any low but fixed T, if the number of players is big enough, then in the long run, 
almost all players use C strategy. On the other hand, if for any fixed number of players, T is lowered 
substantially, then all three strategies appear with frequencies close to 1/2. 

Let us sketch briefly the reason of such a behavior. While it is true that both X B and X c have the 
same potential which is the half of the payoff of the whole system (it plays the role of the total energy 
of a system of interacting particles), the X c Nash configuration has more lowest-cost excitations. 
Namely, one player can change its strategy and switch to either A or B and the total payoff will 
decrease by 8 units. Players in the X B Nash configuration have only one possibility, that is to switch 
to C; switching to A decreases the total payoff by 16. Now, the probability of the occurrence of any 
configuration in the Gibbs state (which is the stationary state of our stochastic dynamics) depends on 
the total payoff in an exponential way. One can prove that the probability of the ensemble consisting 
of the X c Nash configuration and configurations which are different from it at few sites only is much 
bigger than the probability of the analogous X B -ensemble. It follows from the fact that the X c - 
ensemble has many more configurations than the X B -ensemble. On the other hand, configurations 
which are outside X B and X^-ensembles appear with exponentially small probabilities. It means that 
for large enough systems (and small but not extremely small T) we observe in the stationary state the 
X c Nash configuration with perhaps few different strategies. The above argument was made into a 
rigorous proof for an infinite system of the closely related lattice-gas model (the Blume-Capel model) 
of interacting particles by Bricmont and Slawny in [23] . 
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In the above example, X B and X c have the same total payoff but X c has more lowest-cost excita- 
tions and therefore it is low-noise ensemble stable. We will now discuss the situation, where X c has 
a smaller total payoff but nevertheless in the long run C is played with a frequency close to 1 if the 
noise level is low but not extremely low. We will consider a family of games with the following payoff 
matrix: 

Example 4 

ABC 

A 1.5 1 
U = B 2 + a 1 

CI 12, 
where a > so B is both payoff and pairwise risk-dominant. 

We are interested in the long-run behavior of our system for small positive a and low T. One may 
modify the proof of Theorem 1 (see Appendix B) and obtain the following theorem. 

Theorem 2. For every e > 0, there exist a(e) and T(e) such that for every < a < a(e), there 
exists T(a) such that for T(a) < T < T(e), X c is e-ensemble stable, and for < T < T(a), X B is 
e-ensemble stable. 

Observe that for a = 0, both X B and X c are stochastically stable (they appear with the frequency 
1/2 in the limit of zero noise) but X c is low-noise ensemble stable. For small a > 0, X B is both 
stochastically (it appears with the frequency 1 in the limit of zero noise) and low-noise ensemble 
stable. However, for intermediate noise T(a) < T < T(e), if the number of players is big enough, 
then in the long run, almost all players use the strategy C - X c is ensemble stable). If we lower T 
below T(ct), then almost all players start to use the strategy B. T = T{a) is the line of the first-order 
phase transition. In the thermodynamic limit, there exist two Gibbs state (equilibrium states) on this 
line. We may say that at T = T(a), the society of players undergoes a phase transition from C to 
5-behavior. 

Now we will consider games with a dominated strategy and two symmetric Nash equilibria. We 
say that a given strategy is is dominated if it gives a player the lowest payoff regardless of a strategy 
chosen by an opponent. It is easy to see that dominated strategies cannot be present in any Nash 
equilibrium. Therefore such strategies should not be used by players and consequently we might think 
that their presence should not have any impact on the long-run behavior of the system. We will show 
in the following example that this may not be necessarily true. 
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Example 5 





A 


B 


C 


A 





0.1 


1 


B 


0.1 


2 + a 


1 


C 


1 


1 


2, 



where a > 0. 

We see that strategy A is dominated by B and C hence X A is not a Nash equilibrium. X B and 
X c are both Nash equilibria but only X B is a ground-state configuration for —U. In the absence of 
A, B is both payoff and risk-dominant and therefore is stochastically stable and low-noise ensemble 
stable. Adding the strategy A does not change dominance relations; B is still payoff and pairwise 
risk dominant. However, we may modify slightly the proof of Theorem 2 to show that X c is e- 
ensemble stable at intermediate noise levels. The mere presence of the dominated strategy A changes 
the long-run behavior of the system. Similar results were already discussed in adaptive games of 
Myatt and Wallace [25]. In their games, at every discrete moment of time, one of the agents leaves the 
population and is replaced by another one who plays the best response. He calculates his best response 
with respect to his own payoff matrix which is the matrix of a common average payoff disturbed by 
a realization of some random variable with the zero mean. The noise does not appear in the game 
as a result of players' mistakes but is the effect of their idiosyncratic preferences. The authors then 
show that the presence of a dominated strategy may change the stochastic stability of equilibria. 
However, the reason for such a behavior is different in their and in our models. In our model, it is 
relatively easy to get out of X c and this makes X^-ensemble stable. Mayatt and Wallace introduce 
a dominated strategy in such a way that it is relatively easy to make a transition to it from a risk 
and payoff-dominant configuration and then with a high probability the system moves to a third Nash 
configuration which results in its stochastic stability. 

Although in above models, the number of players was very large, their strategic interactions were 
decomposed into a sum of two-player games. Stochastic and ensemble stability of three-player games 
were investigated in [26]. 

5. Stochastic Stability in Non-potential Games 

Let us now consider games with three strategies and three symmetric Nash equilibria: (A, A), (B, B), 
and (C, C). Generically, such games do not have a potential and therefore their stationary states 
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cannot be explicitly constructed. To find them, we must resort to different methods. We will use a 
tree representation of the stationary distribution of Markov chains [20,21] (see also Appendix C). 
To illustrate this technique we will discuss a following two-player game with three strategies. 

Example 6 

Players are located on a finite subset A of Z (with periodic boundary conditions) and interact with 
their two nearest neighbors. They have at their disposal three pure strategies: A, B, and C. The 
payoffs are given by the following matrix: 





A 


B 


c 


A 


3 





2 


B 


2 


2 





C 








3 



Our game has three Nash equilibria, (A,A),(B,B), and (C, C). Let us note that in pairwise 
comparisons, B risk dominates A, C dominates B and A dominates B. The corresponding spatial 
game has three homogeneous Nash configurations: X A ,X B , and X c . They are the only absorbing 
states of the noise-free best-response dynamics. When we start with any state different from X A , X B , 
and X° , then after a finite number of steps we arrive at either X A , X B or X c and then stay there 
forever. It follows from the tree representation of stationary states (see Appendix C) that any state 
different from X A ,X B , and X c , has zero probability in the stationary distribution in the zero-noise 
limit. Moreover, in order to study the zero-noise limit of the stationary distribution, it is enough to 
consider probabilities of transitions between absorbing states. 

Theorem 3. X B is stochastically stable 

Proof: The following are maximal A-tree, B-tree, and C-tree: 



B ->C -» A, C ^ A^ B, A^B^C. 



Let us observe that 



Pb^a = 0(e~ e / T ), 




(5.2) 



Pa^c = 0(e' e / T ), 



(5.3) 



where lirn^o 0(x)/x = 1. 



STATISTICAL MECHANICS OF SPATIAL EVOLUTIONARY GAMES 13 

The theorem follows from the tree characterization of stationary states as described in Appendix 

C. 

X B is stochastically stable because it is much more probable (for low T) to escape from X A and X c 
than from X B . The relative payoffs of Nash configurations are not relevant here (in fact X B has the 
smallest payoff). Let us recall Example 3 of a potential game, where an ensemble-stable configuration 
has more lowest-cost excitations. It is easier to escape from an ensemble-stable configuration than 
from other Nash configurations. 

Stochatic stability concerns single configurations in the zero-noise limit; ensemble stability concerns 
families of configurations in the limit of the infinite number of players. It is very important to 
investigate and comparethese two concepts of stability in nonpotential games. 

Nonpotential spatial games cannot be directly presented as systems of interacting particles. They 
constitute a large family of interacting objects not thoroughly studied so far by methods statistical 
physics. Some partial results concerning stochastic stability of Nash equilibria in nonpotential spatial 
games were obtained in [9-11,26,27]. 

One may wish to say that A risk dominates the other two strategies if it risk dominates them in 
pairwise comparisons. In Example 6, that B dominates A, C dominates B, and finally A dominates 
C. But even if we do not have such a cyclic relation of dominance, a strategy which is pairwise risk- 
dominant may not be stochastically stable [27]. A more relevant notion seems to be that of a global 
risk dominance [28]. We say that A is globally risk dominant if it is a best response to a mixed strategy 
which assigns probability 1/2 to A. It was shown in [10,11] that a global risk-dominant strategy is 
stochastically stable in some spatial games with local interactions. 

A different criterion for stochastic stability was developed by Blume [9] . He showed (using techniques 
of statistical mechanics) that in a game with k strategies A{ and k symmetric Nash equilibria (Ai, Ai), 
i = 1, k and k pure symmetric Nash equlibria, A\ is stochastically stable if 

min^^!, Ai) - U{A n , AJ) > max([/(A n , A n ) - U(A U An)). (5.4) 

n>l n>± 

We may observe that if A\ satisfies the above condition, then it is pairwise risk dominant. 

6. Discussion 

We discussed effects of the number of players and the noise level on the long-run behavior in 
the stochastic dynamics of spatial games. In the so-called potential games with the Boltzmann-type 
updating, stationary states are Gibbs distributions of corresponding lattice-gas models. We used ideas 
and techniques of statistical mechanics to analyze such games. 
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In particular, we were concerned with two limits of our models. In the first one, for a fixed number of 
players, one considers an arbitrarily low level of noise. Then the relevant concept is that of stochastic 
stability of single configurations. For a fixed level of noise, in the limit of the infinite number of 
players, long-run behavior is described by the stability of certain ensembles of configurations. We 
show in several examples that the long-run behavior may be different in these two limiting cases. 

In non-potential games, stationary states cannot be explicitly constructed as before. In order to 
study their zero-noise limits, one may use their tree representation. We illustrated this technique on a 
simple example. Constructing stationary states in non-potential spatial games is an important open 
problem. 

Acknowledgments: I thank Christian Maes and Joseph Slawny for useful conversations. Financial 
support by the Polish Committee for Scientific Research under the grant KBN 5 P03A 025 20 is kindly 
acknowledged. 

Appendix A. 

Here we provide a proof of (4.2). We follow [23] very closely. We begin by defining formally 
restricted ensembles. Let 

n = {A,B,C} z2 

be the configuration space of our model. 

nj| = {X e fi, Xi = B, C for all i G Z 2 and if X t = C, then Xj = B if \i - j\ = 1}, 

= {X € ft, ifXi = A or B, then Xj = C if \i - j\ = 1} 

are the restricted ensembles of configurations of the lowest-cost excitations of X B and X c Nash 
configurations. Observe that X c has many more lowest-cost excitations than X B . 

We define partition functions of restricted ensembles with boundary conditions Y € \c, k = B,C 

as 

Z r (A\Y) = J2^ Ua{X \ (A.l) 
where the sum is over X 6 which are equal to F on A c , 

U A (X)= U ( X *> X j)> ( A - 2 ) 

{i,j}nA^0 

and (3 = 1/T. It is a standard result in rigorous statistical mechanics that a following limit exists 

^ R {(3\k) = \\m 2 log^p- (A.3) 
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and has a convergent expansion. ip R (/3\k) is called a thermodynamic potential of a gas of noninteracting 
lowest-cost excitations. We may write 

logZ R (A\Y) = \A\Pi, R ((3\k)+o(e-W)\6A\,k = B,C, (A.4) 

where 

(3t(j R (f3\B) = 2 + e~^ + 0(e-^), (A.5) 
lty R {p\C) = 2 + 2e- 4/3 + 0(e- 8/3 ). (A.6) 

and 6A is the boundary of A. 

We define ret(X) by ret(X)i = B if X* = C and X, = B for \i — j\ = 1, ret(X)i = C if X* = A, B 
and Xj = C for \i—j\ = 1, and ret(X)i = Xi otherwise. Therefore, in ret(X) we remove all lowest-cost 
excitations of X but not excitations of a higher cost. If X G Q R {^ R ), then ret(X) = X B (X c ). 
Let us define the boundary of X as the set of pairs such that ret{X)i / ret(X)j A small scale 
contour 7 of a configuration A is a pair 7 = ( [7] , A[ 7 j ) , where [7] is the maximal connected subset of 
the union of the boundary of X and pairs of sites such that X; L = Xj = A. The cost of 7 is 

17( 7 ) = £ (2 — U(Xi,Xj)) 

Now we define large-scale contours. Let L(/3) = e 5/3//2 . We cover Z 2 with squares 

B(i) = B(o) + (l/2)Li,i G Z 2 , 

where B(o) is the square of side L{(3) centered at the origin and containing e 5/3 lattice sites. We call 
B{i) a regular box of A if X B ^ G B ^ and it is irregular otherwise. There are two types of irregular 
boxes of X: 

type 1 if X B{i) G fL RB(i) , 

type 2 if a small-scale contour of A intersects B(i). 

A large-scale contour T is a connected family of irregular squares. Let | |T| | be the number of squares 
in T and |T| the number of lattice sites in T, [T] = UserB. For any function / on SI 

„/?E{ l ,j}nA^0 U(Xi,Xj) 

PaV\Y) = £/W , (A.7) 

where the sum is over A G f2 which are equal to Y on A c . For [r] C A, let Pa(F|1^) = P\(xa\Y), 
where x\(X) = 1 if T is a contour of A and zero otherwise. Therefore 

PA(r|y) " Z(A\Y) ' (A ' 8) 

where 

Z(A|r,y) = £e^ A W, (A.9) 
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where the sum is over X € VL which are equal to Y on A c and contain Y. Pp(»\Y) is called a Gibbs 
measure in A with boundary conditions Y. Now we are ready to formulate our main proposition. 

Proposition 1. For big enough (5 there exists c such that for all finite A C Z 2 , all boundary conditions 
Y € Jl^ Ac and all contours T contained in A 
Pa{F\Y) < e-^iril. 

Proof: First we condition on strategies in 5[T], 

P A (T\Y) = Y, PaF\Y, Z)P A (Z\Y). (A.10) 

z 

Then we get 

P A (T\Y,Z) = P [T] (T\Y,Z) = Z f { ^ z y (A.ll) 

Z([T]\T, Y,Z) = J2Y, Z (\nr\", y, Z), (A.12) 

where the first summation is over all possible families T 2 of type-2 squares of T and the second 
over families oo of small-scale contours in [r] such that for each square of V 2 there is a contour of uj 
intersecting the square. Let 

[r] - [u] = U a M a , [u] = U T6a ,[ 7 ] 
be the decomposition of [r] — [uj] into connected components. Now we have 

Z{[T]\T 2 ^,Y,Z)=e 2 ^^\ e ^) IiaZR{Ma \x a ), (A.13) 

where 

f/H = J>( 7 ), (A.14) 

| 7 | is the number of pairs in 7 and X a is the configuration on SM a . 

After inserting (A.13) into (A.12) and (A.12) into (A.ll) we have to estimate the ratio 

(U a Z R (M a \X a )) (U a Z R (M a \X a )) 

Z([T]\Y,Z) ~ Z R ([T}\Y,Z) ' ( ■ " j 

where in the dominator we used the lower bound 

Z([T]\Y,Z)>Z R ([T]\Y,Z). (A.16) 
We write the volume terms of (A. 15) as 

e /3(E a \M a \^ R (f3\k(a))-m R W\C)) < e -(e- 4 P+0(e-W)) E a;fc(a)/ c \ M «\ (A.17) 
< e -(l/2)|r 1 |e- 4 ' 3 = e -(l/2)||r 1 ||e' 3 
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where T 1 = r — T 2 . We also have to estimate boundary terms. The family of boundaries of 5M a 
consist of two subfamilies: one contained in [oj] and another contained in 5[T], on which we have the 
same boundary conditions, Y and Z, in the numerator and the denominator of (A. 15). Since these 
boundary conditions are the same, contributions to the boundary term cancel each other. Finally 
using f/(7) > [7I we obtain that (A. 15) is bounded by 

(A.18) 



3 -(l/2)||r 1 ||e' 3 + C V|e- 4 ' 3 



Therefore 



where 



We obtain that 



Z([r]|r 2 ,^,Y,Z) p>u(u)-{i/2)\\ri\\ee /» 1Q) 

Z R ([T]\Y,Z) ~ e ' lA ' iyj 



ff = P- c'e- 4/3 . (A.20) 



P A (F\Y, Z) < e -(V2)l|rM|e^ ^ e -/WH (A 21) 

where the sum over the families uj of small scale-contours is restricted by the condition that for each 
B G T there exists at least one contour 76W with [7] n B ^ 0. We get 

^ e -^H<n Ber2 (^(l/m!) £ g-^E^O {K22) 

u> m>l 7l>— >7m 

<n Kr ^(i/ m !)£r^WD, 

m>l 7 

where the superscript B indicates summation over contours 7 with [7] n B 7^ 
Now because U{pi) > I7I and U{^f) > 6, for big (3 we get 

J2 e-P'^ < c"\B\e-W' = c"e^. (A.23) 

7 

From (A.22) and (A.23) we get 

(e c " e ^ -l)||r 2 || < ( C "' e -^)H r2 . (A.24) 

We conclude the proof by using the above estimate in (A. 21). 
Now the following proposition is a consequence of Proposition 1 

Proposition 2. There exist two positive constants, c and d , such that Pa(||F|| > c|<5A| \Y) < e <='/3|<5^V| 
for big enough (3 and for all finite A C Z 2 , T in A, and all boundary conditions Y G 

Proof: We change boundary conditions from an arbitrary Y to C. We have 

Pa{»\Y) < e 4l3 ^P A (.\C). (A.25) 
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We connect disconnected parts of T through 8 A and from Proposition 1 we get 
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P A (||r|| > c\5A\ \C) < e - c '^ SA \ (A.26) 

which finishes our proof. 
Proof of (4.2): 

By Proposition 2 we may assume that T covers a small part of A. Indeed, with high probability we 
have 

|r| = e 5/3 \\T\\ < 0(e 5f3 )\5A\. (A.27) 

In the complement of [r] we have the gas of noninteracting lowest-cost excitations of X c which are 
very rare if (3 is large enough so the noise level T = 1/(5 is low enough. This proves that there is the 
unique limit \\m K __, Z 2 P^(»\Y) which is equal to fi T in (4.1) and (4.2) is established. 

Appendix B. 

The payoff of X B in Example 5 is bigger than that of X c . However, for small a, X c has again 
larger thermodynamic potential. Thermodynamic potentials of lowest-cost excitations have following 
expansions: 

I3^ R {I3\B) = 2 + a + e - 4(1+a)l3 + 0(e" 8(1+Q)/3 ). (B.l) 
M R (P\C) = 2 + 2e- 4/3 + 0(e- 8/3 ). (B.2) 

If a < ie" 4/3 , then 

(3^ R (l3\C)-^ R (P\B))>^e-^. (B.3) 
Now to prove Theorem 2 we may repeat the proof of Theorem 1. 

Appendix C. 

The following tree representation of stationary states of Markov chains was proposed by Freidlin and 
Wentzell (1970 and 1984). Let (£l,P) be an irreducible Markov chain with a state space Q and 
transition probabilities given by P : f2 x Q — > [0, 1]. It has a unique stationary probability distribution 
\i (called also a stationary state). For X € Q, an A-tree is a directed graph on f2 such that from every 
Y ^ X there is a unique path to X and there are no outcoming edges out of X. Denote by T(X) the 
set of all A-trees and let 

deT(x) (Y,Y')ed 
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where the product is with respect to all edges of d. Now one can show that 

H{X) = q{X) (C.2) 

for all lei!. 

In our case, P is given by (3.1). A state is an absorbing one if it attracts nearby states in the 
noise-free best-response dynamics. Let us assume that after a finite number of steps of the noise-free 
dynamics we arrive at one of the absorbing states (there are no other recurrence classes) and stay there 
forever. Then it follows from the above tree representation that any state different from absorbing 
states has zero probability in the stationary distribution in the zero-noise limit. Moreover, in order 
to study the zero-noise limit of the stationary state, it is enough to consider paths between absorbing 
states. More precisely, we construct X-trees with absorbing states as vertices; the family of such 
X-trees is denoted by T{X). Let 

q m (X)=max def{x) ]J P{Y,Y% (C3) 
(Y,Y')ed 

where P(Y,Y') = max Y[(ww) P(W, W), where the product is taken along any path joining Y 
with Y' and the maximum is taken with respect to all such paths. Now we may observe that if 
lim e ->oq m (Y)/q m (X) = 0, for any Y ^ X, then X is stochastically stable. Therefore we have to 
compare trees with the biggest products in (C.3); such trees we call maximal. 
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